1. Field of the Invention
The present invention relates to an image processing apparatus and a relevant method, and in particular, that which can be preferably applied to the correction of distortion aberration due to a lens of a digital camera or the like.
Priority is claimed on Japanese Patent Application No. 2005-338406, filed Nov. 24, 2005, the content of which is incorporated herein by reference.
2. Description of the Related Art
It is generally known that an optical system of a camera has a distortion aberration regardless of the type of camera (i.e., a digital camera or a film-based camera). Additionally, most presently-available cameras employ optical zooming in which the state of distortion aberration varies when the lens setting is changed from widest to longest. If the distortion aberration is a barrel-type characteristic, a checkered subject as shown in FIG. 27A is observed as an image shown in FIG. 28A, that is, the observed image has a barrel-type distortion.
Generally, images having such distortion aberration are not undesirable as images captured by cameras. Therefore, various methods for correcting the distortion aberration using image processing have been proposed. With respect to the barrel-type distortion aberration, in a specific example as shown in FIG. 26, positions of a captured image indicated by solid lines are corrected to positions indicated by dashed lines so as to cancel the aberration.
FIG. 25 is a block diagram showing the structure of an example of ordinary image processing apparatuses for performing image processing which includes distortion aberration correction. In FIG. 25, a preprocessing part 1 obtains evaluation values for an image obtained via a CCD (charge coupled device), which are used in processes such as AE (automatic exposure), AF (auto focus), AWB (auto white balancing), and the like. In addition, after performing OB clamping (black level determining process), defect pixel correction, and the like, the preprocessing part 1 stores processed image data into an SDRAM (synchronous dynamic random access memory) 3. A CPU (central processing unit) 2 computes values for exposure and focusing, R, G, and B gains, coefficients for white balancing, and the like, with reference to AE, AF, and AWB values.
The SDRAM 3 is a storage device for temporarily storing image data and the like.
An image processing part 4 extracts image data of a specific rectangular area among the image data stored in the SDRAM 3, and subjects the extracted data to some processes such as white balancing, shading correction, synchronization, filtering, and the like.
A DMA (direct memory access) part 41 as a constituent of the image processing part 4 reads data of the specific rectangular area of the image, and a first image processing part 42 executes image processing which is performed before the distortion correction. The first image processing part 42 includes a structure for performing filtering (specifically, an LPF (low pass filter) process) as shown in FIG. 24. In FIG. 24, a delay part 421 includes an FIFO (first-in first-out) memory, SRAM (static random access memory), and FF (flip flop). The delay part 421 extracts 25 pixels including a target pixel and its peripheral pixels, and a filtering part 422 executes a filtering process by multiplying the 25-pixel data by specific coefficients. This process is executed for each of the R, G, and B signals.
A distortion correcting part 5 subjects the R, G, and B signals input from the image processing part 4 to correction of distortion aberration, and a post-correction (i.e., post-distortion-correction) coordinate generating part 54 in this processing part 5 outputs coordinate data (e.g., of intersections between dashed lines in FIG. 26) on an image output after the distortion correction.
The signal output from the post-correction coordinate generating part 54 is simultaneously supplied to an R distortion correcting part 51, a G distortion correcting part 52, and a B distortion correcting part 53. These processing parts have an identical structure, and the R distortion correcting part 51 will be explained representatively.
A pre-correction (i.e., pre-distortion-correction) coordinate computing part 511 computes coordinate data (e.g., of intersections between solid lines in FIG. 26) on the image before the distortion correction. This coordinate data corresponds the coordinate data output from the post-correction coordinate generating part 54. That is, with respect to the pixel at coordinates (X, Y) on the image after the distortion correction, the pre-correction coordinate computing part 511 computes the corresponding position (X′, Y′) (of this pixel) on the image which has a distortion and is stored in a memory such as the SDRAM 3.
An address conversion part 512 computes address data for acquiring data of 16 peripheral pixels from the memory, which are close to the coordinates (X′, Y′) output from the pre-correction coordinate computing part 511.
A memory 513 has a group of two-port memories for storing color image data.
With respect to the data of 16 peripheral pixels output from the memory 513, an interpolation computing part 514 performs interpolation based on a by-cubic method, so as to output a corrected R signal. In this process, coefficients for interpolation are computed using a decimal part of the coordinates (X′, Y′).
An image processing part 6 subjects the image after the correction of distortion aberration to some processes such as edge enhancement, chroma suppressing, and the like. When signals of a Bayer image are synchronized by executing interpolation which uses peripheral pixels, the chroma suppressing process is performed for suppressing a (pseudo) color which should not appear originally but appears in the vicinity of an edge of the image. In this process, the level of each color signal component of relevant pixels in the vicinity of the edge is gradually suppressed.
The image processing part 6 consists of a second image processing part 61 and a DMA part 62. The second image processing part 61 executes image processing performed after the distortion correction, and the DMA part 62 outputs results of the processing to the SDRAM 3.
FIG. 23 shows an example of the structure of a circuit for performing the chroma suppressing. In FIG. 23, a matrix part 611 converts the R, G, and B signals into Y, Cr, and Cb signals for indicating brightness and color difference by using a 3×3 matrix. An edge determination part 612 is a circuit for determining the vicinity of an edge of the image based on the brightness signal (i.e., the Y signal). A suppress coefficient computing part 613 outputs a suppress coefficient so as to suppress pseudo colors with respect to pixels determined as being close to the edge. A multiplication part 614 multiplies the color component signals Cr and Cb by the coefficient supplied from the suppress coefficient computing part 613.
In accordance with the above structure, a rectangular image “a” having an arc-shape distortion (see dashed lines in FIG. 22A), which was stored from the preprocessing part 1 into the SDRAM 3, is corrected by the distortion correcting part 5 to a rectangular image “b” (see FIG. 22B). In FIG. 22A, the area indicated by a reference symbol “c” is the rectangular area extracted by the DMA part 41.
In the image processing parts 4 and 6 in FIG. 25, conventionally, image processing is executed without consideration of distortion of the image. However, when considering correction of the distortion aberration, as shown in FIG. 26, the correction is performed in a manner such that the central part is contracted while the peripheral part is expanded. Therefore, in comparison with the central part, the peripheral part of the image has less high-frequency signal components and thus is blurred. With respect to this phenomenon, a known image processing technique such as outline correction may be applied to the image before the correction of distortion aberration (see, for example, Japanese Unexamined Patent Application, First Publication No. 2004-272578).
Also in conventional techniques, when a filtering process (5×5) is performed using adjacent pixels in image processing (performed by the image processing part 4 in the example shown in FIG. 25) before the correction of distortion aberration, if the distortion aberration is a barrel type, the adjacent pixels existing in a central part of the image are positioned as shown in 21A after the correction of distortion aberration. In contrast, in a peripheral part having a large distortion aberration, if the distortion aberration is such a barrel type, the adjacent pixels are positioned as shown in 22A after the correction of distortion aberration.